Copilot: surface AI credit usage for token-based-billing seats - #2647
Copilot: surface AI credit usage for token-based-billing seats#2647KSEGIT wants to merge 11 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aaeb632f41
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| else { return nil } | ||
| return CopilotDeviceFlow.makeRequestURL( | ||
| host: CopilotUsageFetcher.apiHost(enterpriseHost: enterpriseHost), | ||
| path: "/orgs/\(encoded)/settings/billing/ai_credit/usage") |
There was a problem hiding this comment.
For the organization AI-credit flow, GitHub's REST docs list this endpoint as GET /organizations/{org}/settings/billing/ai_credit/usage, not /orgs/{org}/... (checked https://docs.github.com/en/rest/billing/usage#get-billing-ai-credit-usage-report-for-an-organization). With the current path, authorized org admins receive a 404 and fetchCreditsUsed returns nil, so the new organization credits row never appears.
Useful? React with 👍 / 👎.
| return 0 | ||
| } | ||
|
|
||
| let creditItems = report.usageItems.filter { $0.unitType == "ai-credits" } |
There was a problem hiding this comment.
For organization AI-credit reports, GitHub's example response uses unitType: "credits", while "ai-credits" is shown for user-level reports (checked https://docs.github.com/en/rest/billing/usage#get-billing-ai-credit-usage-report-for-an-organization). After the path is corrected, real org responses will fall into the “none matched” branch here and return nil, so organization usage still will not render unless this accepts the org unit type.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
Codex review: needs real behavior proof before merge. Reviewed August 12, 2026, 8:09 AM ET / 12:09 UTC. ClawSweeper reviewWhat this changesThis PR adds a per-Copilot-account AI-credit allowance that renders existing seat credit usage as a menu-card progress bar and updates the visible cached row when the allowance changes. Merge readiness⛔ Blocked until stronger real behavior proof is added - 4 items remain Keep open: the seat-credit UI is a useful remaining feature beyond the shipped credit counter, but the current head still lacks real after-fix proof for its cached text-to-bar transition. Priority: P2 Review scores
Verification
How this fits togetherCodexBar fetches Copilot quota data, turns it into provider-detail rows in a usage snapshot, and renders those rows in the menu card. This change supplies a user-entered seat allowance so a known credit counter can be shown as a ratio without inventing a limit. flowchart LR
A[Copilot account] --> B[GitHub usage response]
C[Seat allowance setting] --> D[Credit row builder]
B --> D
D --> E[Cached usage snapshot]
E --> F[Menu card text or progress bar]
Before merge
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Rebase or refresh against current main, then provide a redacted current-head run that shows a real Copilot credit row changing from text to bar when an allowance is entered and back to text when cleared. Do we have a high-confidence way to reproduce the issue? Yes—source and the supplied fixture/live payload establish the path: a token-billed Copilot response with credits_used plus a configured seat allowance produces the detail-row ratio. A real current-head account run is still needed to prove the cache-transition behavior. Is this the best way to solve the issue? Yes in design: an opt-in, per-account denominator avoids fabricating GitHub credit limits while preserving the existing no-fake-percentage guard. It is not yet merge-ready without current-head proof of the cached transition. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against a90dfed5c264. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (26 earlier review cycles; latest 8 shown)
|
|
Thanks for this, and for the excellent original report in #2593 — you found both the |
clawsweeper
Bot
added
rating: 🧂 unranked krab
|
Re The upstream data is real, not inferredLive from a Copilot Business seat (redacted), via the endpoint CodexBar already calls: "quota_snapshots": {
"premium_interactions": { "unlimited": true, "entitlement": 0, "remaining": 0,
"percent_remaining": 100.0, "credits_used": 31,
"token_based_billing": true, "overage_permitted": true },
"chat": { "unlimited": true, "entitlement": 0, "credits_used": 0 },
"completions": { "unlimited": true, "entitlement": 0, "credits_used": 0 }
},
"organization_login_list": ["<ORG>"],
"copilot_plan": "business", "token_based_billing": true,
"quota_reset_date": "2026-09-01"Note And the org endpoint: { "usageItems": [
{ "product": "Copilot", "sku": "Copilot AI Credits", "model": "Code Review model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 31.13 },
{ "product": "Copilot", "sku": "Copilot Cloud Agent", "model": "Coding Agent model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 49.97 } ] }31.13 + 49.97 = 81.10, which matches the "81 / 6,000 AI credits" numerator on the org billing page exactly. That agreement is the strongest evidence I have that the org lane reads the right thing. What the card actually rendersAsserted on exact strings through the real
The org tests drive What I can't evidence yetNo screenshot. Producing one means building this branch, packaging it, and running it against a real Copilot Business account — which per AGENTS.md is exactly the kind of validation that can raise Keychain prompts, so I didn't do it unasked. Happy to if you want it. Two of three surfaces are untouched. I've added a table to the PR description rather than leave it implied: the menu card shows credits; the menu bar icon, the widget, and So today a Business token-billing user still gets an empty icon and has to open the menu. I can wire the icon up in this PR — publishing credits as a Since the PR was opened
|
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
clawsweeper
Bot
added
merge-risk: 🚨 auth-provider
|
After-fix runtime proof — head Menu card (live)
Org lane trace (live)
The best-effort degradation works exactly as designed: warning logged, card unaffected, no fabricated org row, seat lane intact. A finding from the live run: the 404 is a scope issue, not a path issueWith a The Copilot device flow requests only |
|
@clawsweeper re-review |
clawsweeper
Bot
added
proof: sufficient
|
@steipete Hi there, I think it is ready to merge :) Let me know if you want me to do anything more :) |
clawsweeper
Bot
added
rating: 🦐 gold shrimp
Remove the opt-in organization billing lane pending an approved authorization model for org-wide billing reads with device-flow tokens. Rewrite the cached seat credit row immediately when the entitlement changes so a stale denominator or bar never survives a failed refresh.
|
@clawsweeper re-review Addressed the 2026-08-09 14:17 UTC review on head 14599df:
|
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
Text-only credit rows carried no structured usage, so entering an entitlement during an offline or failed refresh left the cached row without its bar. Stamp usageValue on the row at fetch time and use it as the repair numerator.
|
Head 766101c addresses the P2 cached-state finding from the 22:02 UTC review:
After-fix live proof from this head (text→bar transition on my Business seat) is next — will post the screenshot and update the PR body, then request re-review. |
clawsweeper
Bot
added
rating: 🦪 silver shellfish
2 participants
Closes #2593.
The problem
On a Copilot Business seat with
token_based_billing: true,GET /copilot_internal/userreports every quota snapshot asunlimited: true, entitlement: 0, remaining: 0, percent_remaining: 100. The#1258guards correctly drop those so no misleading "0% used" bar appears — but the result is a Copilot card showing only the plan label and no usage at all.Real consumption is available, in two places CodexBar didn't read:
credits_used, already present on each quota snapshot in the response CodexBar already fetches (landed in Decode Copilot credits_used for token-billed seats (#2593) #2613).GET /organizations/{org}/settings/billing/ai_credit/usage— org-wide, per-model (this PR).GitHub publishes no credit entitlement anywhere
This is the finding that shaped the design. I probed all 8 documented billing endpoints plus
budgets,cost-centers,included_credits,ai_credit/entitlement,copilot/metricsandusage/summary. None expose the included-credit ceiling — the "6,000" that the org billing page displays.discountQuantityreveals only what included credits absorbed, so the ceiling is observable only once exceeded.It is derivable as seats × per-seat allowance, but the per-seat figure is currently a promotional 3,000 against a standard 1,900 — a hardcoded table would silently produce a wrong bar when the promo ends. So the denominator is user-entered and never inferred. A lane with no entitlement renders a text row, never a bar with a fabricated ceiling.
If you know of an endpoint I missed, that would simplify this considerably.
What this adds
Seat credit bar (builds on #2613). Extends the shared provider-detail row contract with an optional row id and progress ratio (
ProviderDetailSection.Row), so the existing "Credits used" row becomes a "31 / 3000" bar when a seat entitlement is set, and stays plain text otherwise. Created only when it carries real signal (token-billed / unlimited quota / credits > 0 / entitlement set), so metered accounts reportingcredits_used: 0don't gain a permanent empty row. No guard line inCopilotUsageFetcher.swiftis modified — the#1258guard block is byte-identical to main.Org lane (opt-in, off by default).
GET /organizations/{org}/settings/billing/ai_credit/usage(the documented path), org read fromorganization_login_list.first, summingusageItems[].grossQuantityfiltered to credit unit types —"credits"per the organization-report docs,"ai-credits"per the user-level docs and live org responses; both are accepted, unrelated unit types are rejected (a response with no credit items returnsnil, never a fabricated 0). Strictly best-effort, shaped like the existingaddBudgetWindowsIfNeeded: any failure returns the snapshot untouched. The device flow still requests onlyread:user, so rejection is the expected common case, not an error — it logs a warning rather than failing silently.On my account the org endpoint returns 31.13 + 49.97 = 81.10, matching the billing page's numerator exactly.
Per-account entitlements. Seat and org allowances are stored on the selected
ProviderTokenAccount(following the z.ai per-account fields precedent), resolved incopilotSettingsSnapshotwith the existing global UserDefaults values as fallback — so existing single-account setups keep working unchanged, and two Copilot accounts with different allowances each get the right denominator. The settings fields write to the selected account when one exists.Which surfaces this covers
Worth being explicit, since it's a menu bar app:
codexbar usageCLICredits are detail rows, not a
RateWindow, because a lane with no user-entered entitlement has no percentage to show.MenuBarLayout,IconRemainingResolverand the widget all build fromprimary/secondary/tertiary/extraRateWindows, so they see nothing.The consequence: a Business token-billing user still gets an empty menu bar icon and must open the menu to see credits. I'd rather state that than have you find it.
Making the icon work needs a decision I didn't want to make unilaterally: publishing credits as a
NamedRateWindowwhen an entitlement exists would light up the icon, widget and the existing "menu bar secondary metric" picker for free, but Copilot's extra windows are gated behindcopilotBudgetExtrasEnabledin the card renderer, so it risks either double-rendering the row or coupling credits to an unrelated setting. Happy to do it in this PR if you'll say which shape you want.Testing
make check— 0 violations across 1,811 files.make test— full suite green exceptMiniMaxMenuCardBillingTests, which fails identically on a cleanorigin/maincheckout on this machine (locale-dependent number grouping, "1.234" vs "1,234"); unrelated to this PR.ProviderHTTPTransportStubor the existingURLProtocolstub. No live network, no Keychain access, per AGENTS.md.Regression coverage worth calling out: a test asserts the
#1258behaviour still holds — a Business token-billing payload yields credits andprimary == nil, secondary == nil. New per-account tests cover: account override wins over the global fallback, fallback when the account has no value, two accounts producing different snapshot entitlements, and a Codable round-trip through the config store (including legacy JSON without the new keys).Known limitations
L("AI credits"),L("Org credits"),L("credits used")have noLocalizable.stringsentries.Scripts/check-app-locales.mjsenforces all 23 catalogs together, so a partial addition failsmake check— left out deliberately rather than done badly. Happy to add all 23 here if you'd prefer.organization_login_list.firstwins when a user belongs to several orgs. The row title names the org so it isn't ambiguous, but there's no picker.resetsAtis available and a monthly credit budget is a good fit for it (cf. Show ahead/behind pace for all providers and in the menu bar #807, Add Projections to Copilot Tracker #2166).Commits
9 focused commits, each with its own tests, left unsquashed so they can be reviewed in sequence. Squash on merge if you'd rather.
🤖 Generated with Claude Code